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•The  paper  attempts  to  show  how  the  emerging  field  of  Cognitive  Science  can 
have  a major  impact  on  practical  education.  In  particular  it  provides  an 
overview  of  two  proiects  at  Bolt,  Beranek,  (£)  Newman:  the  work,  on  Socratic 
tutoring  by  Collins  and  Stevens  and  the  work  on  diagnosing  student's  mis- 
conceptions in  arithmetic  by  Brown  and  Burton.  These  two  projects  are  citec 
as  examples  of  how  the  techniques  of  Cognitive  Science  make  it  possible  to 
analyze  teaching  and  learning  in  genuinely  novel  ways. 
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ABSTRACT 


The  paper  attempts  to  show  how  the  emerging  field  of  Cognitive  Science  can  have 
a major  impact  on  practical  education.  In  particular  it  provides  an  overview 
of  two  projects  at  Bolt,  Beranek  & Newman:  the  work  on  Socratic  tutoring  by 
Collins  and  Stevens  and  the  work  on  diagnosing  student's  misconceptions  in 
arithmetic  by  Brown  and  Burton.  These  two  projects  are  cited  as  examples 
of  how  the  techniques  of  Cognitive  Science  make  it  possible  to  analyze  teach- 
ing and  learning  in  genuinely  novel  ways. 


EXPLICATING  THE  TACIT  KNOWLEDGE  IN  TEACHING  & LEARNING 


Allan  Collins 

The  question  I decided  to  address  is  whether  Cognitive 
Science  has  anything  new  to  say  about  education,  given  that  some 
of  the  world's  best  minds  have  been  thinking  about  education 
for  thousands  of  years.  The  answer  I think  is  yes;  Cognitive 
Science  provides  a set  of  theoretical  formalisms  and  analysis 
techniques  that  can  be  used  to  study  teaching  and  learning  of 
topics  like  math  and  sciences  in  a genuinely  novel  way.  My 
plan  is  to  show  how  this  is  possible  with  two  case  studies: 
first  I will  outline  some  work  Stevens  and  I (Collins,  1977; 
Stevens  & Collins,  1977;  Stevens,  Goldin  & Collins,  in  press) 
have  done  on  tutoring  by  the  Socratic  method,  and  second  I will 
describe  work  of  Brown  & Burton  (1978)  on  diagnosing  students' 
procedural  errors  in  arithmetic.  These  are  only  examples  of  the 
kind  of  analyses  possible. 

In  our  work  on  Socratic  tutoring,  we  analyzed  dialogues  by 
a number  of  tutors  teaching  different  subjects  (medicine,  geo- 
graphy, etc.)  using  the  Socratic  method.  The  Socratic  method  is 
made  up  of  a variety  of  individual  strategies.  These  strategies 
involve  entrapping  the  student  into  mistakes  of  different  kinds, 
confronting  him  with  counterexamples,  forcing  him  to  make  predic 
tions,  etc.  Generally  the  method  requires  the  student  to  derive 
general  principles  from  specific  cases,  and  learn  to  use  these 
principles  to  make  predictions  about  new  cases. 
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Using  the  production  rule  formalism  developed  by  Newell  & 
Simon  (1972),  it  was  possible  to  characterize  the  different  stra- 
tegies tutors  use  in  terms  of  rules  of  the  form  "when  in  situa- 


tion X,  do  Y."  We  can  give  a few  examples  to  show  what  these 
' rul es  1 ook  1 i ke : 

Rule  1:  Form  a general  rule  for  an  insufficient  factor. 

I 

(1)  the  student  gives  as  an  explanation  one  or  more  factors  that 
| are  not  sufficient, 

then 

& (2)  formulate  a general  rule  asserting  that  the  factor  given  is 

■ sufficient  and  ask  the  student  if  the  rule  is  true. 

Example:  If  the  student  gives  water  as  the  reason  they  grow  rice 

■ in  China,  ask  him  "Do  you  think  any  place  with  enough  water  can 

I grow  rice?" 

■ Rule  2:  Pick  a counterexample  for  an  insufficient  factor. 

If 

| (1)  the  student  gives  as  an  explanation  one  or  more  factors  that 

are  not  sufficient,  or 

1 (2)  agrees  to  the  general  rule  in  Rule  1, 

■ then 

(3)  pick  a counterexample  that  has  the  right  value  of  the  factor(s) 

I given,  but  the  wrong  value  of  the  dependent  variable,  and 

(4)  ask  what  the  value  of  the  dependent  variable  is  for  that  case, 

I or  (5)  ask  why  the  casual  dependence  does  not  hold  for  that  case. 

I 
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Example:  If  a student  gives  water  as  the  reason  they  grow  rice 

in  China  or  agrees  that  any  place  with  enough  water  can  grow  rice, 
pick  a place  like  Ireland  where  there  is  enough  water  and  ask, 

"Do  they  grow  rice  in  Ireland?"  or  "Why  don't  they  grow  rice  in 
Ireland?" 

Rule  3:  Question  a prediction  made  without  enough  information.: 
If 

(1)  a student  makes  a prediction  as  to  the  value  of  the  depen- 
dent variable  on  the  basis  of  some  set  of  factors,  and 

(2)  there  is  another  value  consistent  with  that  set  of  factors, 
then 

(3)  ask  the  student  why  not  the  other  value. 

Example:  If  the  student  predicts  they  grow  wheat  in  Nigeria  be- 

cause it  is  fertile  and  warm,  ask  him  why  not  rice. 

While  in  one  sense  the  Socratic  method  is  a single  approach 
that  involves  teaching  the  student  to  reason  from  cases,  in 
another  sense  it  is  made  up  of  a variety  of  these  specific  stra- 
tegies that  good  teachers  hit  upon  in  the  course  of  their  teach- 
ing. Some  hit  upon  one  set,  some  upon  another,  though  there  is 
usually  some  overlap.  There  is  little  need  for  teachers  to  ver- 
balize these  strategies,  since  their  application  only  depends  on 
an  intuitive  feel  as  to  how  to  use  them.  If  they  are  taught, 
they  are  usually  taught  by  example.  So  there  is  no  very  specific 
body  of  knowledge  about  the  Socratic  method,  and  hence  there  is 
no  theory  to  be  extended  and  refined.  In  fact  until  computers 
provided  us  with  formalisms  for  expressing  "process  models," 

3 


i 


it  is  unlikely  that  anyone  would  have  thought  of  constructing  a 
specific  theory  about  such  a thing  as  the  Socratic  method. 

By  making  the  tacit  knowledge  about  Socratic  teaching  expli- 
cit, it  becomes  possible  to  educate  teachers  about  the  Socratic 
method.  We  can  teach  them  the  strategies  that  many  different 
teachers  have  developed  and  the  situations  where  they  apply.  If  they 
were  to  evolve  the  method  on  their  own  or  from  watching  other 
good  teachers,  they  would  at  best  develop  only  a subset  of  these 
strategies. 

More  generally  the  method  of  analysis  can  be  applied  to 
analyze  the  specific  strategies  used  by  the  most  effective 
teachers  in  the  country.  Such  an  analysis  can  be  derived  from 
videotapes  of  actual  classes.  The  analysis  would  determine  the 
teacher's  systematic  patterns  of  response  to  particular  teaching 
situations.  It  is  a kind  of  formal  analysis  that  simply  was  not 
possible  in  terms  of  earlier  formalisms  in  psychology,  such  as 
S-R  bonds  or  stage  models.  Simply  put  we  can  now  construct  a 
formal  theory  of  the  strategies  used  by  our  most  successful 
teachers,  and  we  can  thereby  make  their  accumulated  tacit  know- 
ledge available  to  every  potential  teacher. 

My  second  example  concerns  modelling  the  learner  to  diag- 
nose his  misunderstandings.  In  a system  called  BUGGY,  Brown  & 

Burton  (1978)  developed  a representation  called  a procedural  net- 
work in  which  they  represent  all  the  procedures  necessary  to 
carryout  addition  and  subtraction  in  explicit  detail.  In  BUGGY, 
procedural  errors  are  represented  as  perturbations  of  the 
correct  procedures.  In  this  way  the  program  can  simulate  any 
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consistent  procedure  that  students  follow  even  if  it  is  a wrong 
procedure,  as  in  the  example  below. 

Consider  five  "snap  shots"  of  a student's  performance  doing 
addition  as  might  be  seen  on  a homework  assignment.  Before  pro- 
ceeding, discover  the  student's  bug. 

Sample  of  the  student's  work: 

41  328  989  66  216 

+9  +917  +52  +887  +13 

50  1345  1141  1053  229 

Once  you  have  discovered  the  bug,  try  testing  your  hypothesis  by 
"simulating"  the  buggy  student  so  as  to  predict  his  results  on 
the  following  two  test  problems. 

446  201 

+815  +399 

The  bug  is  really  quite  simple.  In  computer  terms,  the 
student,  after determi ni ng  the  carry,  forgets  to  reset  the  "carry 
register"  to  zero  and  hence  the  amount  carried  is  accumulated 
across  the  columns.  For  example,  in  the  student's  second  prob- 
lem, (e.g.,  328  + 917  equals  1345)  he  proceeds  as  follows: 

8+7  = 15  so  he  writes  5 and  carries  1,  2+1  = 3 plus  the  one 
carry  is  4,  lastly  3+9  = 12  but  that  one  carry  from  the  first 
column  is  still  there  --  it  hasn't  been  reset  --  so  adding  it 
in  to  this  column  gives  13.  If  this  is  the  bug,  then  the  answers 
to  the  test  problems  will  be  1361  and  700.  This  bug  is  not  so 
absurd  when  one  considers  that  a child  might  use  his  fingers  to 
remember  the  carry  and  forget  to  bend  back  his  fingers,  or 
counters,  after  each  carry  is  added. 
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It  turns  out  that  it  is  very  difficult  for  experts  to  diag- 
nose systematic  errors  that  BUGGY  makes,  especially  when  the  er- 
rors are  in  low  level  procedures  that  are  called  by  different 
high  level  procedures,  or  when  there  is  more  than  one  bug  in  dif- 
ferent procedures.  To  a teacher  the  students  errors  are  likely 
to  look  random. 

One  way  the  BUGGY  program  can  be  used  is  to  train  teachers 
and  students  how  to  diagnose  bugs.  The  program  selects  a bug, 
and  gives  a few  examples  of  the  bug  as  manifested  in  working 
problems.  The  user  then  selects  problems  for  BUGGY  to  work 
until  he  thinks  he  has  figured  out  the  bug.  Then  BUGGY  presents 
him  with  five  test  cases  where  he  must  simulate  the  bug.  If  the 
user  cannot  produce  the  same  answer  as  BUGGY  on  any  of  the  test 
cases  he  then  goes  back  to  trying  to  diagnose  the  bug.  In  this 
way  teachers  can  be  taught  how  to  be  expert  diagnosticians. 

More  importantly  Brown  & Burton  have  used  BUGGY  to  analyze 
automatically  a subtraction  test  taken  by  1300  students  in  the 
4th  to  6th  grades.  They  found  that  about  40%  of  the  students 
used  essentially  correct  procedures,  about  20%  have  what  appear 
to  be  random  errors  to  BUGGY,  and  about  40%  have  systematic  pro- 
cedural errors.  From  their  data  they  have  compiled  a list  of  the 
20  most  frequent  errors  in  subtraction  and  their  frequencies 
among  grade  school  students.  The  three  most  common  bugs  are  the 
following: 

1.  Borrow  from  zero  (e.g.,  103-45  = 158)  Frequency  107/1300 
When  borrowing  from  a column  whose  top  digit  is  0,  the 
student  writes  9,  but  does  not  continue  borrowing  from  the 
column  to  the  left  of  the  0. 
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2.  Smaller  from  larger  (e.g.,  253-118  = 145)  Frequency  = 54/1300 
The  student  subtracts  the  smaller  digit  in  a column  from  the 
larger  digit  regardless  of  which  one  is  on  top. 

3.  Difference  of  O-N  = N and  jump  over  zero  in  borrowing 
(e.g.,  204-25  = 129)  F req uency= 34/ 1 300 

When  ever  the  top  digit  in  a column  is  0,  the  student  writes 
the  bottom  digit  in  the  answer;  i.e.,  0-N  = N.  When  the 
student  needs  to  borrow  from  a column  whose  top  digit  is 
0,  he  skips  that  column  and  borrows  from  the  next  one. 

The  implications  of  BUGGY  for  testing  are  profound.  No 
longer  need  a test  be  a means  to  assign  a score  to  a student. 

A score  is  an  arbitrary  measure  that  tells  very  little  about 
how  well  the  student  is  doing  or  what  his  real  problem  is.  For 
example,  a single  bug  in  a low  level  procedure  will  produce 
many  more  wrong  answers  than  several  bugs  in  higher  level  pro- 
cedures. What  a test  becomes,  if  BUGGY  is  taken  seriously,  is 
a method  for  diagnosing  the  students  underlying  errors.  The 
outcome  is  a statement  of  whatever  specific  procedural  errors 
the  student  is  making.  Then  teaching  can  be  directed  explicitly 
at  the  bugs  the  test  has  diagnosed. 

Surely  before  BUGGY  arithmetic  had  been  studied  very  thor- 
oughly. But  by  applying  a Cognitive  Science  analysis,  Brown 
& Burton  could  find  new  insights  into  the  problems  children 
are  having.  Other  basic  skills  (higher  mathematics,  reading, 
writing,  etc.)  are  more  difficult  to  analyze  in  this  way,  but 
the  lesson  should  be  obvious.  There  is  enormous  leverage  in 
this  kind  of  analysis  and  the  potential  implications  for  prac- 
tical education  are  profound. 


7 


REFERENCES 


Brown,  J.S.  & Burton,  R.R.  Diagnostic  models  for  procedural  bugs 
in  mathematics.  Cognitive  Science,  1978,  176-204. 


Collins,  A.M.  Processes  in  acquiring  knowledge.  In  R.C.  Anderson, 
R.J.  Spiro,  & W.E.  Montague  (eds.),  Schooling  & the  acquisition 
of  knowledge.  Hillsdale,  N.J.:  Lawrence  Erlbaum  Associates, 

1 977. 


Stevens,  A.L.  & Collins,  A.M.  The  goal  structure  of  a Socratic 
tutor.  Proceedings  of  the  Association  for  Computary  Machinery 
National  Conference.  Seattle,  Washington,  October,  1977. 


Stevens,  A.L.,  Goldin,  S.,  & Collins,  A.M.  Diagnosing  students 
misconceptions  in  causal  models.  BBN  Report,  March,  1978. 


I 


8 


Dr.  Marshall  J.  Farr,  Director 
Personnel  & Training  Research  Programs 
Office  of  Navy  Research  (Code  458) 
Arlington,  VA  22217 

ONR  Branch  Office 
495  Summer  Street 
Boston,  MA  02210 
Attn:  Dr.  James  Lester 

ONR  Branch  Office 
1030  East  Green  Street 
Pasadena,  CA  91 101 
Attn:  Dr.  Eugene  Gloye 

ONR  Branch  Office 
536  S.  Clark  Street 
Chicago,  IL  60605 
Attn:  Dr.  Charles  E.  Davis 

Dr.  M.A.  Bert in,  Scientific  Director 
Office  of  Naval  Research 
Scientific  Liaison  Group/Tokyo 
American  Embassy 
AP0  San  Francisco,  CA  96503 

Office  of  Naval  Research 
Code  200 

Arlington,  VA  22217 

Commanding  Officer 
Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20190 

Director,  Human  Resource  Management 
Naval  Amphibious  School 
Naval  Amphibious  Base,  Little  Creek 
Norfolk,  VA  23521 

LCDR  Charles  J.  Theisen,  Jr.,  MSC,  USN 
4024 

Naval  Air  Development  Center 
Warminster,  PA  18974 

Commanding  Officer 

U.S.  Naval  Amphibious  School 

Coronado,  CA  92155 

Commanding  Officer 
Naval  Health  Research  Center 
San  Diego,  CA  92152 
Attn:  Library 

Chairman,  Leadership  & Law  Dept. 

Div.  of  Professional  Development 
U.S.  Naval  Academy 
Annapolis,  MD  21402 


Scientific  Advisor  to  the  Chief 
of  Naval  Personnel  (Pers  Or) 

Naval  Bureau  of  Personnel 
Room  4410,  Arlington  Annex 
Washington,  DC  20370 

Dr.  Jack  R.  Borsting 
Provost  & Academic  Dean 
U.S.  Naval  Postgraduate  School 
Monterey,  CA  93940 

Mr.  Maurice  Callahan 
N0DAC  (Code  2) 

Dept,  of  the  Navy 

Bldg.  2,  Washington  Navy  Yard 

( Anacostia) 

Washington,  DC  20374 

Office  of  Civilian  Personnel 
Code  342/02  WAP 
Washington,  DC  20390 
Attn:  Dr.  Richard  J.  Niehaus 

Office  of  Civilian  Personnel 
Code  263 

Washington,  DC  20390 

Superintendent  (Code  1424) 

Naval  Postgraduate  School 
Monterey,  CA  93940 

Mr.  George  N.  Graine 
Naval  Sea  Systems  Command 
SEA  047C12 

Washington,  DC  20362 

Chief  of  Naval  Technical  Training 
Naval  Air  Station  Memphis  (75) 
Millington,  TN  38054 
Attn:  Dr.  Norman  J.  Kerr 

Principal  Civilian  Advisor 
for  Education  and  Training 
Naval  Training  Command,  Code  00A 
Pensacola,  FL  32508 
Attn:  Dr.  William  L.  Maloy 

Dr.  Alfred  F.  Smode,  Director 
Training  Analysis  & Evaluation  Group 
Department  of  the  Navy 
Orlando,  FL  32813 

Chief  of  Naval  Education  and 
Training  Support  (01A) 

Pensacola,  FL  32509 


Capt.  H.J.  Connery,  USN 
Navy  Medical  R&D  Command 
NNMC,  Bethesda , MD  20014 

Navy  Personnel  R&D  Center 
Code  01 

San  Diego,  CA  92152 

Navy  Personnel  R&D  Center 
Code  306 

San  Diego,  CA  92152 
Attn:  Dr.  James  McGrath 

A. A.  Sjoholm,  Head,  Technical  Support 
Navy  Personnel  R&D  Center 
Code  201 

San  Diego,  CA  92152 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 
Attn:  Library 

Dr.  J.D.  Fletcher 
Advanced  Research  Projects  Agency 
Cybernetics  Technology  Office 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

Capt.  D.M.  Gragg,  MC,  USN 
Head,  Section  on  Medical  Education 
Uniformed  Services  Univ.  of 
the  Health  Sciences 
6917  Arlington  Road 
Bethesda,  MD  20014 

Officer-in-Charge 
Navy  Occupational  Development  & 
Analysis  Center  (N0DAC) 

Building  150,  Washington  Navy  Yard 
( Anacostia) 

Washington,  DC  20374 

LCDR  J.W.  Snyder,  Jr. 

F-14  Training  Model  Manager 
VF-124 

San  Diego,  CA  92025 

Dr.  John  Ford 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152 


Technical  Director 

U.S.  Army  Research  Institute  for  the 
Behavioral  & Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Armed  Forces  Staff  College 
Norfolk,  VA  23511 
Attn:  Library 

Commandant 

U.S.  Army  Infantry  School 
Fort  Benning,  GA  31905 
Attn:  ATSH-I-V-IT 

Commandant 

U.S.  Army  Institute  of  Administration 
Fort  Benjamin  Harrison,  IN  46216 
Attn:  EA 

Dr.  Beatrice  Farr 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Frank  J.  Harris 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Ralph  Dusek 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Leon  Nawrocki 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Joseph  Ward 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Milton  S.  Katz,  Chief 
Individual  Training  & Performance 
Evaluation  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Col.  G.B.  Howard 
U.S.  Army 

Training  Support  Activity 
Fort  Eustis,  VA  23604 


1 

1 

I 


Dr.  Worth  Scanland 

Chief  of  Naval  Education  & Training 

NAS,  Pensacola,  FL  32508 


I 


Col.  Frank  Hart,  Director 
Training  Management  Institute 
U.S.  Army,  Bldg.  1725 
Fort  Eustis,  VA  23604 

HQ  USAREUE  & 7th  Army 
0DCS0PS 

USAREUR  Director  of  GED 
APO  New  York  09403 

ARI  Field  Unit  - Leavenworth 

P.0.  Box  3122 

Ft.  Leavenworth,  KS  66027 

DCDR,  USAADMINCEN 
Bldg.  #1,  A310 

Ft.  Benjamin  Harrison,  IN  46216 
Attn:  AT21-0ED  Library 

Dr.  Edgar  Johnson 

U.S.  Army  Research  Institute 

1300  Wilson  Blvd, 

Arlington,  VA  22209 

Dr.  James  Baker 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Research  Branch 
AFMPC/DPMYP 

Randolph  AFB,  TX  78148 

AFHRL/AS  (Dr.  G.A.  Eckstrand) 
Wright-Patterson  AFB 
Ohio  45433 

Dr.  Ross  L.  Morgan  (AFHRL/ASR) 
Wright-Patterson  AFB 
Ohio  45433 

Dr.  Marty  Rockway  (AFHRL/TT) 
Lowry  AFB 
Colorado  80230 

Instructional  Technology  Branch 
AFHRL 

Lowry  AFB,  CO  80230 

Dr.  Alfred  R.  Fregly 
AFOSR/NL , Building  410 
Bolling  AFB,  DC  20332 

Dr.  Sylvia  R.  Mayer  (MCIT) 

HQ  Electronic  Systems  Division 
LG  Hanscom  Field 
Bedford,  MA  01730 


Capt.  Jack  Thorpe,  USAF 
AFHRL/FTS 

Williams  AFB,  AZ  85224 

Air  University  Library 
AUL/LSE  76-443 
Maxwell  AFB,  AL  36112 

Dr.  T.E.  Cotterman 
AFHRL/ASR 

Wright  Patterson  AFB 
Ohio  45433 

Dr.  Donald  E.  Meyer 
U.S.  Air  Force 
ATC/XPTD 

Randolph  AFB,  TX  78148 
Dr.  Wilson  A.  Judd 

McDonnell-Douglas  Astronautics  Co.  East 
Lowry  AFB 
Denver,  CO  80230 

Dr.  William  Strobie 

McDonnell  Douglas  Astronautics  Co.  East 
Lowry  AFB 
Denver,  CO  80230 

Director,  Office  of  Manpower 
Utilization 

HQ,  Marine  Corps  (Code  MPU) 

BCB,  Building  2009 
Quantico,  VA  22134 

Dr.  A.L.  Slafkosky 
Scientific  Advisor  (Code  RD-1) 

HQ,  U.S.  Marine  Corps 
Washington,  DC  20380 

AC/S,  Education  Programs 
Education  Center,  MCDEC 
Quantico,  VA  22134 

Mr.  Joseph  J.  Cowan,  Chief 
Psychological  Research  Branch  (G-P-1/62) 
U.S.  Coast  Guard  Headquarters 
Washington,  DC  20590 

Advanced  Research  Projects  Agency 
Administ  *itive  Services 
1400  Wilson  Blvd. 

Arlington,  VA  22209 
Attn:  Ardella  Holloway 

Dr.  Harold  F.  O'Neil,  Jr. 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 


T' 


1 


i 


I 

I 

I 

I 

I 

I 

I 

I 


Dr.  Stephen  Andriole 

Advanced  Research  Projects  Agency 

1400  Wilson  Blvd. 

Arlington,  VA  22209 

Defense  Documentation  Center 
Cameron  Station,  Bldg.  5 
Alexandria,  VA  22314 

Military  Assistant  for  Human  Resources 
Office  of  the  Director  of  Defense 
Research  & Engineering 
Room  3D 129,  The  Pentagon 
Washington,  DC  20301 

Director,  Management  Information 
Systems  Office 
OSD,  MARA 

Room  3B917,  The  Pentagon 
Washington,  DC  20301 

Dr.  Vern  Urry 
Personnel  RAD  Center 
U.S.  Civil  Service  Commission 
1900  E Street  NW 
Washington,  DC  20415 

Dr.  Andrew  R.  Molnar 
Science  Education  Dev.  A Res. 

National  Science  Foundation 
Washington,  DC  20550 

Dr.  Marshall  S.  Smith 
Associate  Director 
NIE/OPEPA 

National  Institute  of  Education 
Washington,  DC  20208 

Dr.  Joseph  L.  Young,  Director 
Memory  A Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 

Dr.  H.  Wallace  Sinaiko,  Director 
Manpower  Research  A Advisory  Services 
Smithsonian  Institution 
801  N.  Pitt  Street 
Alexandria,  VA  22314 

Dr.  James  M.  Ferstl 
Employee  Development : Training 
Technologist 
Bureau  of  Training 
U.S.  Civil  Service  Commission 
Washington,  DC  20415 

Dr.  Ronald  P.  Carver 

School  of  Education 

University  of  Missouri-Kansas  City 

5100  Rockhill  Road 

Kansas  City,  MO  64110 


William  J.  McLaurin 
Room  301 

Internal  Revenue  Service 
2221  Jefferson  Davis  Hwy. 
Arlington,  VA  22202 


Dr.  John  R.  Anderson 
Dept,  of  Psychology 
Yale  University 
New  Haven,  CT  06520 

Dr.  Scarvia  B.  Anderson 
Educational  Testing  Service 
Suite  1040 

3445  Peachtree  Road  NE 
Atlanta,  GA  30326 

Professor  Earl  A.  Alluisi 
Code  287 

Dept,  of  Psychology 
Old  Dominion  University 
Norfolk,  VA  23508 

Dr.  Daniel  Alpert 
Computer-Based  Education 
Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

Ms.  Carole  A.  Bagley 
Applications  Analyst 
Minnesota  Educational 
Computing  Consortium 
1925  Sather  Ave. 

Lauderdale,  MN  55113 

Dr.  John  Brackett 
SofTech 

460  Totten  Pond  Road 
Waltham,  MA  02154 

Dr.  Robert  K.  Branson 
1A  Tully  Bldg. 

Florida  State  University 
Tallahassee,  FL  32306 

Dr.  John  Seely  Brown 

Bolt  Beranek  and  Newman  Inc. 

50  Moulton  Street 
Cambridge,  MA  02138 

Dr.  Victor  Bunderson 

Institute  for  Computer  Uses  in  Education 
355  EDLC 

Brigham  Young  University 
Provo,  UT  84601 


I 


. r%t 


I 

I 


Jacklyn  Caselli 

ERIC  Clearinghouse  on  Information 
Resources 

Stanford  University 
School  of  Education  - SCRDT 
Stanford,  CA  9U305 

Century  Research  Corporation 
4113  Lee  Highway 
Arlington,  VA  22207 

Dr.  A.  Charnes 
BEB  20 3E 

University  of  Texas 
Austin,  TX  78712 

Dr.  Kenneth  E.  Clark 
College  of  Arts  & Sciences 
University  of  Rochester 
River  Campus  Station 
Rochester,  NY  14627 

Dr.  John  J.  Collins 
Essex  Corporation 
6305  Caminito  Estrellado 
San  Diego,  CA  92120 

Dr.  Ruth  Day 
Dept,  of  Psychology 
Yale  University 
2 Hillhouse  Avenue 
New  Haven,  CT  06520 

ERIC  Facility-Acquisitions 
4833  Rugby  Avenue 
Bethesda , MD  20014 

Dr.  John  Eschenbrenner 
McDonnell  Douglas  Astronautics 
Company-East 
P.0.  Box  30204 
St.  Louis,  MO  80230 

Major  I.N.  Evonic 
Canadian  Forces  Personnel 
Applied  Research  Unit 
1107  Avenue  Road 
Toronto,  Ontario,  CANADA 

Dr.  Victor  Fields 
Dept,  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

Dr.  Edwin  A.  Fleishman 
Advanced  Research  Resources  Organization 
8555  Sixteenth  Street 
Silver  Spring,  MD  20910 


Dr.  Larry  Francis 
University  of  Illinois 
Computer-Based  Educational  Research  Lab 
Champaign,  IL  61801 

Dr.  Frederick  C.  Frick 
MIT  Lincoln  Laboratory 
Room  D 268 
P.0.  Box  73 
Lexington,  MA  02173 

Dr.  John  R.  Frederiksen 
Bolt  Beranek  and  Newman  Inc. 

50  Moulton  Street 
Cambridge,  MA  02138 

Dr.  Vernon  S.  Gerlach 
College  of  Education 
146  Payne  Bldg.  B 
Arizona  State  University 
Tempe,  AZ  85281 

Dr.  Robert  Glaser,  Co-Director 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

Dr.  M.D.  Havron 

Human  Sciences  Research,  Inc. 

7710  Old  Spring  House  Road 
West  Gate  Industrial  Park 
McLean,  VA  22101 

Dr . Duncan  Hansen 
School  of  Education 
Memphis  State  University 
Memphis,  TN  3811 8 

Human  Resources  Research  Organization 
400  Plaza  Bldg. 

Pace  Blvd.  at  Fairfield  Drive 
Pensacola,  FL  32505 

HumRRO/Western  Division 
27857  Berwick  Drive 
Carmel,  CA  93921 
Attn:  Library 

HumRRO/Columbus  Office 

Suite  23,  2601  Cross  Country  Drive 

Columbus,  GA  31 906 

HumRRO/Ft.  Knox  Office 

P.0.  Box  293 

Fort  Knox,  KY  40121 


HumRRO/Western  Division 
27857  Berwick  Drive 
Carmel,  CA  93921 
Attn:  Dr.  Robert  Vineberg 

Dr.  Lawrence  B.  Johnson 

Lawrence  Johnson  & Associates,  Inc. 

Suite  502 

2001  S Street  NW 

Washington,  DC  20009 

Dr.  Arnold  F.  Kanarick 
Honeywell,  Inc. 

2600  Ridgeway  Pkwy. 

Minneapolis,  MN  55413 

Dr.  Roger  A.  Kauftoan 
203  Dodd  Hall 
Florida  State  University 
Tallahassee,  FL  32306 

Dr.  Steven  W.  Keele 
Dept,  of  Psychology 
University  of  Oregon 
Eugene,  OR  97403 
Dr.  David  Klahr 
Dept,  of  Psychology 
Carnegie-Mellon  University 
Pittsburgh,  PA  15213 

Mr.  W.E.  Lassiter 
Data  Solutions  Corp. 

Suite  21 1 , 6849  Old  Dominion  Drive 
McLean,  VA  22101 

Dr.  Robert  R.  Mackie 
Human  Factors  Research,  Inc. 

6780  Corton  Drive 

Santa  Barbara  Research  Park 

Goleta,  CA  93017 

Dr.  William  C.  Mann 
University  of  So.  California 
Information  Sciences  Institute 
4676  Admiralty  Way 
Marina  Del  Rey,  CA  90291 

Dr.  Eric  McWilliams 
Program  Manager 
Technology  and  Systems,  TIE 
National  Science  Foundation 
Washington,  DC  20550 

Dr.  Leo  Munday 
Houghton  Mifflin  Co. 
p.O.  Box  1970 
Iowa  City,  IA  52240 


Dr.  Donald  A.  Norman 
Dept,  of  Psychology  C-009 
University  of  California,  San  Diego 
La  Jolla,  CA  92093 

Mr.  A. J.  Pesch,  President 
Eclectech  Associates,  Inc. 

P.O.  Box  178 

N.  Stonington,  CT  06359 

Dr.  Kenneth  A.  Polycyn 
PCR  Information  Science  Co. 
Communication  Satellite  Applications 
7600  Old  Springhouse  Rd. 

McLean,  VA  22101 

Dr.  Steven  M.  Pine 
N 660  Elliott  Hall 
University  of  Minnesota 
75  East  River  Road 
Minneapolis,  MN  55455 

R.  Dir.  M.  Rauch 
P II  4 

Bundesministerium  der  Verteidigung 
Postfach  161 
53  Bonn  1,  GERMANY 

Dr.  Joseph  W.  Rigney 
University  of  So.  California 
Behavioral  Technology  Laboratories 
3717  South  Grand 
Los  Angeles,  CA  90007 

Dr.  Andrew  M.  Rose 
American  Institutes  for  Research 
1055  Thomas  Jefferson  St.  NW 
Washington , DC  20007 

Dr.  Leonard  L.  Rosenbaum,  Chairman 
Dept,  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

Dr.  Mark  D.  Reckase 
Educational  Psychology  Dept. 
University  of  Missouri-Columbia 
12  Hill  Hall 
Columbia,  M0  65201 

Dr.  Robert  J.  Seidel 
Instructional  Technology  Group, 
HumRRO 

300  N.  Washington  St. 

Alexandria,  VA  22314 


Dr.  Persia  Sturgis 
Dept,  of  Psychology 
California  State  University-Chico 
Chico,  CA  95926 

Mr.  Dennis  J.  Sullivan 
c/o  Canyon  Research  Group,  Inc. 
32107  Lindero  Canyon  Road 
Westlake  Village,  CA  91360 

Mr.  Walt  W.  Tornow 
Control  Data  Corporation 
Corporate  Personnel  Research 
p.0.  Box  0 - HQN060 
Minneapolis,  MN  55440 

Dr.  Benton  J.  Underwood 
Dept,  of  Psychology 
Northwestern  University 
Evanston,  IL  60201 

Dr.  Carl  R.  Vest 
Battelle  Memorial  Institute 
Washington  Operations 
2030  M Street  NW 
Washington,  DC  20036 

Dr.  David  J.  Weiss 
Dept,  of  Psychology 
N660  Elliott  Hall 
University  of  Minnesota 
Minneapolis,  MN  55455 

Dr.  Keith  Wescourt 
Dept,  of  Psychology 
Stanford  University 
Stanford,  CA  94305 

Dr.  Claire  E.  Weinstein 
Educational  Psychology  Dept. 
University  of  Texas  at  Austin 
Austin,  TX  78712 

Dr.  Anita  West 
Denver  Research  Institute 
University  of  Denver 
Denver,  CO  80201 

Mr.  Thomas  C.  O'Sullivan 
TRAC 

1220  Sunset  Plaza  Drive 
Los  Angeles,  CA  90069 

Dr.  Richard  Snow 
Stanford  University 
School  of  Education 
Stanford,  CA  94305 


Dr.  Thomas  G.  Sticht 
Assoc.  Director,  Basic  Skills 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

Prof.  Fumiko  Samejima 
Dept,  of  Psychology 
Austin  Peay  Hall  304C 
University  of  Tennessee 
Knoxville,  TN  37916 

Liam  Bannon 
Dept,  of  Psychology 
University  of  Western  Ontario 
London , Ontario , 

CANADA 

Mr.  Daniel  Bobrow 

Xerox  Parc 

3333  Coyote  Hill  Rd. 

Palo  Alto,  CA  94306 

Joost  Breuken 

Universitei:  van  Amsterdam 

C0W0 

Spui21 

AMSTERDAM 

Robert  Davis 
University  of  Illinois 
Urbana-Champaign,  ILL  6 1801 

Dr.  Dorothy  Deringer 
National  Science  Foundation 
Division  of  Science  Education 
Development  & Research 
Washington , DC 

John  Gaschnig 

Carnegie-Mellon  University 
Dept,  of  Computer  Science 
Schenley  Park 
Pittsburgh,  PA  15213 

Ira  Goldstein 

The  Artificial  Intelligence  Laboratory 
545  Technology  Square 
Cambridge,  MA  02139 

Jim  Greeno 

University  of  Pittsburgh 

Learning  Research  & Development  Center 

Pittsburgh,  PA 


